Analysis of ambient light for gaze tracking

ABSTRACT

Technologies for gaze tracking by ambient light include a mobile compute device to capture a first image of a user of the mobile compute device by a first camera and a second image of a real-world environment of the mobile compute device by a second camera. The mobile compute device determines a physical location of a light source relative to the mobile compute device based on the second image and identifies a first and second corneal reflection in an eye of the user captured in the first image. The mobile compute device determines, based on the physical location, a first correspondence between the first corneal reflection and the light source and a second correspondence between the second corneal reflection and an image displayed on a display of the mobile compute device. Further, the mobile compute device performs gaze tracking based on the first correspondence and the second correspondence.

BACKGROUND

Eye tracking and gaze tracking techniques are used to determine thedirection of a person's gaze (i.e., the direction the person is looking)based on captured images. In doing so, a wide array of image analysistechniques may be employed. For example, in some embodiments, videoimages may be analyzed to determine the orientation of a person's headand/or the relative position of the person's pupil. Other common gazetracking methods rely on the measurement of reflections of a knowninfrared (IR) light source on the cornea of the person being tracked.Such techniques generally include a dedicated IR camera (e.g., IRprojector and sensor), which may require a significant cost and/orfootprint on a mobile computing device.

BRIEF DESCRIPTION OF THE DRAWINGS

The concepts described herein are illustrated by way of example and notby way of limitation in the accompanying figures. For simplicity andclarity of illustration, elements illustrated in the figures are notnecessarily drawn to scale. Where considered appropriate, referencelabels have been repeated among the figures to indicate corresponding oranalogous elements.

FIG. 1 is a simplified block diagram of at least one embodiment of amobile compute device for gaze tracking by ambient light;

FIG. 2 is a simplified block diagram of at least one embodiment of anenvironment of the mobile compute device of FIG. 1;

FIGS. 3-4 is a simplified flow diagram of at least one embodiment of amethod for gaze tracking by ambient light that may be executed by themobile compute device of FIG. 1;

FIG. 5 is an illustrative image of a real-world environment captured byan environment-facing camera of the mobile compute device of FIG. 1;

FIG. 6 is an illustrative image of a user captured by a user-facingcamera of the mobile compute device of FIG. 1; and

FIG. 7 is a close-up view of the user's eye captured in the illustrativeimage of FIG. 6.

DETAILED DESCRIPTION OF THE DRAWINGS

While the concepts of the present disclosure are susceptible to variousmodifications and alternative forms, specific embodiments thereof havebeen shown by way of example in the drawings and will be describedherein in detail. It should be understood, however, that there is nointent to limit the concepts of the present disclosure to the particularforms disclosed, but on the contrary, the intention is to cover allmodifications, equivalents, and alternatives consistent with the presentdisclosure and the appended claims.

References in the specification to “one embodiment,” “an embodiment,”“an illustrative embodiment,” etc., indicate that the embodimentdescribed may include a particular feature, structure, orcharacteristic, but every embodiment may or may not necessarily includethat particular feature, structure, or characteristic. Moreover, suchphrases are not necessarily referring to the same embodiment. Further,when a particular feature, structure, or characteristic is described inconnection with an embodiment, it is submitted that it is within theknowledge of one skilled in the art to effect such feature, structure,or characteristic in connection with other embodiments whether or notexplicitly described. Additionally, it should be appreciated that itemsincluded in a list in the form of “at least one A, B, and C” can mean(A); (B); (C): (A and B); (B and C); (A and C); or (A, B, and C).Similarly, items listed in the form of “at least one of A, B, or C” canmean (A); (B); (C): (A and B); (B and C); (A and C); or (A, B, and C).

The disclosed embodiments may be implemented, in some cases, inhardware, firmware, software, or any combination thereof. The disclosedembodiments may also be implemented as instructions carried by or storedon one or more transitory or non-transitory machine-readable (e.g.,computer-readable) storage medium, which may be read and executed by oneor more processors. A machine-readable storage medium may be embodied asany storage device, mechanism, or other physical structure for storingor transmitting information in a form readable by a machine (e.g., avolatile or non-volatile memory, a media disc, or other media device).

In the drawings, some structural or method features may be shown inspecific arrangements and/or orderings. However, it should beappreciated that such specific arrangements and/or orderings may not berequired. Rather, in some embodiments, such features may be arranged ina different manner and/or order than shown in the illustrative figures.Additionally, the inclusion of a structural or method feature in aparticular figure is not meant to imply that such feature is required inall embodiments and, in some embodiments, may not be included or may becombined with other features.

Referring now to FIG. 1, a mobile compute device 100 for gaze trackingby ambient light is shown. In use, as described in more detail below,the mobile compute device 100 is configured to capture an image of auser of the mobile compute device 100 and an image of a real-worldenvironment of the mobile compute device 100. The mobile compute device100 further analyzes the captured image of the real-world environment todetermine the physical location of a light source outputting ambientlight visible to the mobile compute device 100 (e.g., not occluded andpowerful enough). Further, in the illustrative embodiment, the mobilecompute device 100 analyzes the captured image of the user to identifycorneal reflections (i.e., glints) in an eye of the user and determinewhich light source each corneal reflection corresponds with (e.g., areflection of a light source captured in the real-world image, areflection of an image displayed on a display of the mobile computedevice 100, etc.) and performs gaze tracking based on thosedeterminations.

The mobile compute device 100 may be embodied as any type of computingdevice capable of performing the functions described herein. Forexample, the mobile compute device 100 may be embodied as a smartphone,cellular phone, wearable computing device, personal digital assistant,mobile Internet device, tablet computer, netbook, notebook, Ultrabook™,laptop computer, and/or any other mobile computing/communication device.Although the mobile compute device 100 is described herein as beingmobile in the illustrative embodiment, it should be appreciated that themobile compute device 100 may be embodied as a stationary computingdevice in other embodiments (e.g., a desktop computer). As shown in FIG.1, the illustrative mobile compute device 100 includes a processor 110,an input/output (“I/O”) subsystem 112, a memory 114, a data storage 116,a communication circuitry 118, a camera system 120, and a display 122.Further, in some embodiments, the mobile compute device 100 may includeone or more sensors 124. Of course, the mobile compute device 100 mayinclude other or additional components, such as those commonly found ina typical computing device (e.g., various input/output devices and/orother components), in other embodiments. Additionally, in someembodiments, one or more of the illustrative components may beincorporated in, or otherwise form a portion of, another component. Forexample, the memory 114, or portions thereof, may be incorporated in theprocessor 110 in some embodiments.

The processor 110 may be embodied as any type of processor capable ofperforming the functions described herein. For example, the processor110 may be embodied as a single or multi-core processor(s), digitalsignal processor, microcontroller, or other processor orprocessing/controlling circuit. Similarly, the memory 114 may beembodied as any type of volatile or non-volatile memory or data storagecapable of performing the functions described herein. In operation, thememory 114 may store various data and software used during operation ofthe mobile compute device 100 such as operating systems, applications,programs, libraries, and drivers. The memory 114 is communicativelycoupled to the processor 110 via the I/O subsystem 112, which may beembodied as circuitry and/or components to facilitate input/outputoperations with the processor 110, the memory 114, and other componentsof the mobile compute device 100. For example, the I/O subsystem 112 maybe embodied as, or otherwise include, memory controller hubs,input/output control hubs, firmware devices, communication links (i.e.,point-to-point links, bus links, wires, cables, light guides, printedcircuit board traces, etc.) and/or other components and subsystems tofacilitate the input/output operations. In some embodiments, the I/Osubsystem 112 may form a portion of a system-on-a-chip (SoC) and beincorporated, along with the processor 110, the memory 114, and othercomponents of the mobile compute device 100, on a single integratedcircuit chip.

The data storage 116 may be embodied as any type of device or devicesconfigured for short-term or long-term storage of data such as, forexample, memory devices and circuits, memory cards, hard disk drives,solid-state drives, or other data storage devices. The data storage 116and/or the memory 114 may store various data during operation of themobile compute device 100 as described herein.

The communication circuitry 118 may be embodied as any communicationcircuit, device, or collection thereof, capable of enablingcommunications between the mobile compute device 100 and other remotedevices over a network (not shown). For example, in some embodiments,the mobile compute device 100 may offload one or more of the functionsdescribed herein to a remote computing device. The communicationcircuitry 118 may be configured to use any one or more communicationtechnologies (e.g., wireless or wired communications) and associatedprotocols (e.g., Ethernet, Bluetooth®, Wi-Fi®, WiMAX, LTE, 5G, etc.) toeffect such communication.

The camera system 120 includes a plurality of cameras configured tocapture images or video (i.e., collections of images or frames) andcapable of performing the functions described herein. It should beappreciated that each of the cameras of the camera system 120 may beembodied as any peripheral or integrated device suitable for capturingimages, such as a still camera, a video camera, or other device capableof capturing video and/or images. In the illustrative embodiment, thecamera system 120 includes a user-facing camera 126 and anenvironment-facing camera 128. Depending on the particular embodiment,each of the user-facing camera 126, the environment-facing camera 128,and/or other cameras of the camera system 120 may be embodied as atwo-dimensional (2D) camera (e.g., an RGB camera) or a three-dimensional(3D) camera. Such 3D cameras include, for example, depth cameras,bifocal cameras, and/or cameras otherwise capable of generating a depthimage, channel, or stream. In another embodiment, one or more of thecameras of the camera system 120 include at least two lenses andcorresponding sensors configured to capture images from at least twodifferent viewpoints of a scene (e.g., a stereo camera).

As described in greater detail below, the user-facing camera 126 isconfigured to capture images of the user of the mobile compute device100. In particular, the user-facing camera 126 captures images of theuser's face, which may be analyzed to determine the location of theuser's eye(s) relative to the mobile compute device 100 (e.g., relativeto the user-facing camera 126, relative to the display 122, and/oranother reference point of the mobile compute device 100). Theenvironment-facing camera 128 captures images of the real-worldenvironment of the mobile compute device 100. In the illustrativeembodiment, the user-facing camera 126 and the environment-facing camera128 are positioned on opposite sides of the mobile compute device 100and therefore have fields of view in opposite directions. In particular,in the illustrative embodiment, the user-facing camera 126 is on thesame side of the mobile compute device 100 as the display 122 such thatthe user-facing camera 126 may capture images of the user as she viewsthe display 122 and a displayed image may be reflected in the user'seye.

The display 122 of the mobile compute device 100 may be embodied as anytype of display on which information may be displayed to a user of themobile compute device 100. Further, the display 122 may be embodied as,or otherwise use any suitable display technology including, for example,a liquid crystal display (LCD), a light emitting diode (LED) display, acathode ray tube (CRT) display, a plasma display, a touchscreen display,and/or other display technology. Although only one display 122 is shownin the illustrative embodiment of FIG. 1, in other embodiments, themobile compute device 100 may include multiple displays 122.

As shown in FIG. 1, the mobile compute device 100 may include one ormore sensors 124 configured to generate data/signals indicative of anenvironment or context of the mobile compute device 100 and/or user ofthe mobile compute device 100. In various embodiments, the sensors 124may be embodied as, or otherwise include, for example, inertial sensors,position sensors, location sensors, proximity sensors, optical sensors,light sensors, audio sensors, temperature sensors, motion sensors,piezoelectric sensors, and/or other types of sensors. Of course, themobile compute device 100 may also include components and/or devicesconfigured to facilitate the use of the sensor(s) 124.

Referring now to FIG. 2, in use, the mobile compute device 100establishes an environment 200 for gaze tracking by ambient light. Theillustrative environment 200 includes a light analysis module 202, aneye analysis module 204, a gaze tracking module 206, and a displaymodule 208. Additionally, in the illustrative embodiment, the eyeanalysis module 204 includes a reflection location module 210 and apupil location module 212. The various modules of the environment 200may be embodied as hardware, software, firmware, or a combinationthereof. For example, the various modules, logic, and other componentsof the environment 200 may form a portion of, or otherwise beestablished by, the processor 110 or other hardware components of themobile compute device 100. As such, in some embodiments, one or more ofthe modules of the environment 200 may be embodied as circuitry orcollection of electrical devices (e.g., a light analysis circuitry, aneye analysis circuitry, a gaze tracking circuitry, a display circuitry,a reflection location circuitry, and/or a pupil location circuitry).Additionally, in some embodiments, one or more of the illustrativemodules may form a portion of another module and/or one or more of theillustrative modules may be independent of one another.

The light analysis module 202 is configured to analyze one or moreimages of the real-world environment captured by the environment-facingcamera 128 to determine the physical location of one or more lightsources outputting ambient light (e.g., a lamp, ceiling light,television, display, etc.) relative to the mobile compute device 100. Asdescribed below, in doing so, the light analysis module 202 maydetermine the image location (e.g., pixels/region in a captured image)of the one or more light sources in the captured image and determine thedirection of the light source(s) relative to the mobile compute device100 based on the image location. For example, in some embodiments, thelight analysis module 202 may identify region(s) of the captured imagehaving high intensity and/or high contrast pixel values relative tosurrounding image regions. As described in greater detail below, thelight analysis module 202 may analyze multiple images of the same lightsource (e.g., from different perspectives) and/or utilize triangulationtechniques to determine the physical location of that light sourcerelative to the mobile compute device 100.

The eye analysis module 204 is configured to analyze one or more imagesof the user captured by the user-facing camera 126. As indicated above,in the illustrative embodiment, the eye analysis module 204 includes thereflection location module 210 and the pupil location module 212. Thereflection location module 210 is configured to analyze a captured imageof the user to identify the user's eye in the captured image and, morespecifically, to identify the corneal reflections in the user's eye. Forexample, the reflection location module 210 may determine the imagelocation in the captured image in which reflections of the ambient lightfrom the light source(s) and/or displayed images on the display 122 arevisible on the user's cornea. As described herein, in some embodiments,the corneal reflections may be identified as regions of the capturedimage having high intensity and/or high contrast pixel values relativeto surrounding image regions. Further, in the illustrative embodiment,the reflection location module 210 is configured to determine whichlight source each corneal reflection corresponds with (e.g., areflection of a light source captured in the real-world image, areflection of an image displayed on the display 122, etc.). In someembodiments, to do so, the reflection location module 210 may match oneor more of the corneal reflections to the light source or displayedimage based on known characteristics of the light source or displayedimage (e.g., size, color, shape, etc.). The pupil location module 212 isconfigured to analyze the image of the user's eye to determine the imagelocation of the user's pupil or, more specifically, the edge of theuser's pupil.

The gaze tracking module 206 is configured to perform gaze tracking(i.e., monitor the direction of the user's gaze) based on the determinedcorrespondences. In some embodiments, the gaze tracking is further basedon the edge of the user's pupil. It should be appreciated that the gazetracking module 206 may utilize any suitable techniques, algorithms,and/or mechanisms for performing gaze tracking consistent with thetechniques described herein.

The display module 208 renders images on the display 122 for the user ofthe mobile compute device 100 to view. In some embodiments, the display122 may be used as an additional source of light for performing theanalyses described herein. Further, in some embodiments, the relativeintensity of the displayed image(s) may be pre-calculated or determined,for example, in order to distinguish the corneal reflections on theuser's eye and determine the proper correspondences between thereflections and the light sources. For example, a high intensity bluelight displayed on the display 122 would be similarly reflected in theuser's cornea (i.e., as a high intensity blue light).

Referring now to FIG. 3, in use, the mobile compute device 100 mayexecute a method 300 for gaze tracking by ambient light. Theillustrative method 300 begins with block 302 in which the mobilecompute device 100 captures a plurality of images with the camera system120. In doing so, in block 304, the mobile compute device 100 capturesone or more images of the real-world environment with theenvironment-facing camera 128. As described herein, the real-worldenvironment may include one or more ambient light sources visible to themobile compute device 100 that may be used for gaze tracking. Forexample, as shown in FIG. 5, an illustrative captured image 500 of thereal-world environment depicts a light source 502. As described below,in some embodiments, the mobile compute device 100 may capture multipleimages of the real-world environment (e.g., with the same camera 128 ormultiple cameras 128) for analysis in determining the physical locationof the light source (e.g., the light source 502) relative to the mobilecompute device 100. In block 306, the mobile compute device 100 capturesone or more images of the user's face with the user-facing camera 126(see, for example, an illustrative captured image 600 as shown in FIG.6).

In block 308, the mobile compute device 100 analyzes the image of thereal-world environment to identify a light source in the captured imageand the direction of the light source relative to the mobile computedevice 100 (e.g., relative to a particular reference point of the mobilecompute device 100). In doing so, in block 310, the mobile computedevice 100 may identify the light source as a region of the capturedimage having high intensity and/or high contrast pixel values relativeto surrounding image regions. For example, a light source that emitswhite ambient light may appear as a bright white region in the capturedimage. In some embodiments, the mobile compute device 100 may utilizeone or more edge detection and/or image segmentation techniques toidentify the light source (e.g., Sobel filters, Canny edge detection,pyramid segmentation, etc.). It should be appreciated that the mobilecompute device 100 may utilize any suitable techniques, algorithms,and/or mechanisms for determining the direction of the light sourcerelative to the mobile compute device 100 based on the identified imagelocation of the light source in the captured image. For example, in someembodiments, the mobile compute device 100 may leverage storedcoordinate information, angular information, and/or other relevantinformation regarding the camera system 120 of the mobile compute device100.

In block 312, the mobile compute device 100 determines the physicallocation of the light source relative to the mobile compute device 100(or other reference point). It should be appreciated that the mobilecompute device 100 may utilize any suitable techniques for doing so. Forexample, in block 314, the mobile compute device 100 may determine thelocation of the light source based on triangulation over multiple imagesof the real-world environment that include that particular light source.In particular, in block 316, the mobile compute device 100 may determinethe physical location based on images of the light source captured bymultiple environment-facing cameras 128 or based on images of the lightsource captured by multiple lenses of the same environment-facing camera128 (e.g., a 3D or stereo camera). In block 318, the mobile computedevice 100 may determine the physical location based on multiple imagesof the light sources captured by the same environment-facing camera 128(or multiple environment-facing cameras 128) at different positionsrelative to the light source (e.g., captured at different points intime). For example, in some embodiments, the environment-facing camera128 may be embodied as a video camera, and motion of the camera 128 maybe estimated (e.g., from the video stream or inertial sensors) in orderto perform triangulation over frames captured at different times andlocations of the mobile compute device 100. In other words, the mobilecompute device 100 may determine the physical location of the lightsource based on the direction of the light source relative to twodifferent reference points of the mobile compute device 100 (e.g., viatriangulation). In other embodiments, in block 320, the mobile computedevice 100 may approximate the distance of the light source from themobile compute device 100 to be a predefined distance (e.g., infinity,an arbitrarily large distance, etc.). It should be appreciated that, insome embodiments, distances exceeding a threshold distance (e.g., twometers) may be estimated to be the predefined distance without sufferingsignificant accuracy loss of gaze estimation.

In some embodiments, the mobile compute device 100 may utilize multipleambient light sources to perform gaze tracking. In such embodiments, inblock 322, the mobile compute device 100 may determine whether toidentify another light source in the captured image(s) of the real-worldenvironment. If so, the method 300 returns to block 308 to analyze theimage(s). Otherwise, the method 300 advances to block 324 of FIG. 4 inwhich the mobile compute device 100 may determine an image displayed onthe display 122 at the time the image of the user was taken. Forexample, in some embodiments, the display 122 may include a displaycontroller that tracks various characteristics of displayed imagesand/or the displayed images themselves, which may be used for variousanalyses as described herein (e.g., matching corneal reflections to thecorresponding light source).

In block 326, the mobile compute device 100 analyzes the captured imageof the user (e.g., the image 600 of FIG. 6) to locate the cornealreflections in the user's eye (e.g., the reflections of the lightsource(s) and/or the images shown on the display 122 from the user'seyes). It should be appreciated that the mobile compute device 100 mayutilize any suitable image analysis techniques, algorithms, filters,and/or mechanisms to locate the corneal reflections, match the cornealreflections with the light sources, and/or perform other analysesdescribed herein. In block 328, the mobile compute device 100 mayidentify the user's eye in the captured image, for example, to reducethe region of the image required to be analyzed for glint analysis andgaze tracking. Although the techniques are described herein in referenceto the analysis of a single eye of the user, it should be appreciatedthat both of the user's eyes may be analyzed in some embodiments. Insome embodiments, in block 330, the mobile compute device 100 mayidentify the corneal reflections on the user's eye as regions of thecaptured image having high intensity and/or high contrast pixel valuesrelative to surrounding image regions.

In block 332, the mobile compute device 100 may determine which lightsource each identified corneal reflection on the user's eye correspondswith (e.g., a reflection of a light source captured in the real-worldimage, a reflection of an image displayed on the display 122, etc.)based on the reflection characteristics. In some embodiments, the mobilecompute device 100 may match each corneal reflection with the source ofthe reflection based on the location, size, color, shape, and/or othercharacteristics of the reflection. For example, as shown in FIG. 7, theuser's eye 700 captured in the illustrative image 600 (see FIG. 6) ofthe user shows two corneal reflections: a corneal reflection 702 of thelight source 502 in the real-world environment of the mobile computedevice 100 (see FIG. 5) and a corneal reflection 704 of the display 122of the mobile compute device 100. It should be appreciated that themobile compute device 100 may utilize any suitable characteristics ofthe corneal reflections and/or other known information to distinguishthe corneal reflections from one another and determine the light sourcewith which they correspond. For example, in many embodiments, thereflection from the display 122 is likely to appear on or near theuser's pupil due to being held directly in front of the user, whereasother ambient light sources are likely to appear in the periphery of theuser's eye, for example, because they would otherwise be occluded by themobile compute device 100. Further, as indicated above, a cornealreflection of the image shown on the display 122 will have similarcharacteristics as the displayed image itself (e.g., a similar shape andcolor).

In block 334, the mobile compute device 100 analyzes the captured imageof the user's eye to determine the image location of the user's pupilor, more specifically, the edge of the user's pupil (e.g., the edge 706as shown in FIG. 7). The mobile compute device 100 may employ anysuitable image analysis techniques for doing so. Further, it should beappreciated that the identification of the user's pupil may independentof the glint analysis techniques described herein and, therefore, mayoccur contemporaneously with, prior to, or subsequent to suchtechniques. In block 336, the mobile compute device 100 performs gazetracking (i.e., monitors the direction of the user's gaze) based on thedetermined correspondences between light sources and corneal reflectionsand the identified edge of the user's pupil. It should be appreciatedthat the gaze tracking may be based on any suitable techniques,algorithms, and/or mechanisms consistent with the techniques describedherein.

In block 338, the mobile compute device 100 determines whether tocontinue gaze tracking. If so, the method 300 returns to block 302 ofFIG. 3 in which the mobile compute device 100 captures another set ofimages with the camera system 120. In other words, in performing gazetracking, the mobile compute device 100 may repeatedly identify thelocation of the light sources relative to the mobile compute device 100(e.g., if the mobile compute device 100 and/or the light sources aremoving) and the corneal reflections that correspond with those lightsources.

EXAMPLES

Illustrative examples of the technologies disclosed herein are providedbelow. An embodiment of the technologies may include any one or more,and any combination of, the examples described below.

Example 1 includes a mobile compute device for gaze tracking by ambientlight, the mobile compute device comprising a display; a camera systemcomprising a first camera and a second camera, the camera system tocapture (i) a first image of a user of the mobile compute device withthe first camera and (ii) a second image of a real-world environment ofthe mobile compute device with the second camera; a light analysismodule to determine a physical location of a light source relative tothe mobile compute device based on the second image; an eye analysismodule to (i) identify a first corneal reflection and a second cornealreflection in an eye of the user and (ii) determine, based on thephysical location, a first correspondence between the first cornealreflection and the light source and a second correspondence between thesecond corneal reflection and an image displayed on the display; a gazetracking module to perform gaze tracking based on the firstcorrespondence and the second correspondence.

Example 2 includes the subject matter of Example 1, and wherein thelight analysis module is to (i) analyze the second image to determine animage location of the light source captured in the second image and (ii)determine a direction of the light source relative to the mobile computedevice based on the image location; and wherein to determine thephysical location of the light source comprises to determine thephysical location based on the determined direction.

Example 3 includes the subject matter of any of Examples 1 and 2, andwherein to analyze the second image comprises to identify a region ofthe second image having at least one of high intensity or high contrastrelative to surrounding regions of the second image.

Example 4 includes the subject matter of any of Examples 1-3, andwherein to determine the physical location comprises to determine thephysical location of the light source based on the second image and athird image of the real-world environment that includes the lightsource.

Example 5 includes the subject matter of any of Examples 1-4, andwherein determining the physical location comprises determining adistance of the light source from the mobile compute device byperforming triangulation based on the second image and the third image.

Example 6 includes the subject matter of any of Examples 1-5, andwherein to determine the physical location comprises to approximate adistance of the light source from the mobile compute device to apredefined distance.

Example 7 includes the subject matter of any of Examples 1-6, andwherein to identify the first corneal reflection and the second cornealreflection comprises to analyze the first image to identify the user'seye.

Example 8 includes the subject matter of any of Examples 1-7, andwherein to identify the first corneal reflection and the second cornealreflection comprises to identify a first region and a second region ofthe first image having at least one of high intensity or high contrastrelative to surrounding regions of the first image.

Example 9 includes the subject matter of any of Examples 1-8, andwherein to determine the first correspondence and the secondcorrespondence comprises to determine the first correspondence and thesecond correspondence based on known characteristics of at least one ofthe light source or the image displayed on the display.

Example 10 includes the subject matter of any of Examples 1-9, andfurther including a display module to determine the image displayed, ata point in time at which the first image is captured, on the display ofthe mobile compute device; wherein to determine the secondcorrespondence comprises to determine the second correspondence based onat least one characteristic of the image displayed.

Example 11 includes the subject matter of any of Examples 1-10, andwherein to determine the second correspondence comprises to determinethe second correspondence based on at least one of a size or color ofthe image displayed.

Example 12 includes the subject matter of any of Examples 1-11, andwherein the eye analysis module is to analyze the first image toidentify an edge of a pupil of the user; and wherein to perform the gazetracking comprises to perform the gaze tracking based further on theedge of the pupil of the user.

Example 13 includes the subject matter of any of Examples 1-12, andwherein the first camera has a field of view in a direction opposite afield of view of the second camera about the display.

Example 14 includes a method for gaze tracking by ambient light, themethod comprising capturing, by a first camera of a mobile computedevice, a first image of a user of the mobile compute device; capturing,by a second camera of the mobile compute device different from the firstcamera, a second image of a real-world environment of the mobile computedevice; determining, by the mobile compute device, a physical locationof a light source relative to the mobile compute device based on thesecond image; identifying, by the mobile compute device, a first cornealreflection and a second corneal reflection in an eye of the usercaptured in the first image; determining, by the mobile compute deviceand based on the physical location, a first correspondence between thefirst corneal reflection and the light source and a secondcorrespondence between the second corneal reflection and an imagedisplayed on a display of the mobile compute device; and performing, bythe mobile compute device, gaze tracking based on the firstcorrespondence and the second correspondence.

Example 15 includes the subject matter of Example 14, and furtherincluding analyzing, by the mobile compute device, the second image todetermine an image location of the light source captured in the secondimage; and determining, by the mobile compute device, a direction of thelight source relative to the mobile compute device based on the imagelocation; wherein determining the physical location of the light sourcecomprises determining the physical location based on the determineddirection.

Example 16 includes the subject matter of any of Examples 14 and 15, andwherein analyzing the second image comprises identifying a region of thesecond image having at least one of high intensity or high contrastrelative to surrounding regions of the second image.

Example 17 includes the subject matter of any of Examples 14-16, andwherein determining the physical location comprises determining thephysical location of the light source based on the second image and athird image of the real-world environment that includes the lightsource.

Example 18 includes the subject matter of any of Examples 14-17, andwherein determining the physical location comprises determining adistance of the light source from the mobile compute device byperforming triangulation based on the second image and the third image.

Example 19 includes the subject matter of any of Examples 14-18, andwherein determining the physical location comprises approximating adistance of the light source from the mobile compute device to apredefined distance.

Example 20 includes the subject matter of any of Examples 14-19, andwherein identifying the first corneal reflection and the second cornealreflection comprises analyzing the first image to identify the user'seye.

Example 21 includes the subject matter of any of Examples 14-20, andwherein identifying the first corneal reflection and the second cornealreflection comprises identifying a first region and a second region ofthe first image having at least one of high intensity or high contrastrelative to surrounding regions of the first image.

Example 22 includes the subject matter of any of Examples 14-21, andwherein determining the first correspondence and the secondcorrespondence comprises determining the first correspondence and thesecond correspondence based on known characteristics of at least one ofthe light source or the image displayed on the display.

Example 23 includes the subject matter of any of Examples 14-22, andfurther including determining the image displayed, at a point in time atwhich the first image is captured, on the display of the mobile computedevice; wherein determining the second correspondence comprisesdetermining the second correspondence based on at least onecharacteristic of the image displayed.

Example 24 includes the subject matter of any of Examples 14-23, andwherein determining the second correspondence comprises determining thesecond correspondence based on at least one of a size or color of theimage displayed.

Example 25 includes the subject matter of any of Examples 14-24, andfurther including analyzing, by the mobile compute device, the firstimage to identify an edge of a pupil of the user; wherein performing thegaze tracking comprises performing the gaze tracking based further onthe edge of the pupil of the user.

Example 26 includes the subject matter of any of Examples 14-25, andwherein the first camera has a field of view in a direction opposite afield of view of the second camera about the display.

Example 27 includes a compute device comprising a processor; and amemory having stored therein a plurality of instructions that whenexecuted by the processor cause the compute device to perform the methodof any of Examples 14-26.

Example 28 includes one or more machine-readable storage mediacomprising a plurality of instructions stored thereon that in responseto being executed result in a compute device performing the method ofany of Examples 14-26.

Example 29 includes a compute device comprising means for performing themethod of any of Examples 14-26.

Example 30 includes a mobile compute device for gaze tracking by ambientlight, the mobile compute device comprising means for capturing, by afirst camera of the mobile compute device, a first image of a user ofthe mobile compute device; means for capturing, by a second camera ofthe mobile compute device different from the first camera, a secondimage of a real-world environment of the mobile compute device; meansfor determining a physical location of a light source relative to themobile compute device based on the second image; means for identifying afirst corneal reflection and a second corneal reflection in an eye ofthe user captured in the first image; means for determining, based onthe physical location, a first correspondence between the first cornealreflection and the light source and a second correspondence between thesecond corneal reflection and an image displayed on a display of themobile compute device; and means for performing gaze tracking based onthe first correspondence and the second correspondence.

Example 31 includes the subject matter of Example 30, and furtherincluding means for analyzing the second image to determine an imagelocation of the light source captured in the second image; and means fordetermining a direction of the light source relative to the mobilecompute device based on the image location; wherein the means fordetermining the physical location of the light source comprises meansfor determining the physical location based on the determined direction.

Example 32 includes the subject matter of any of Examples 30 and 31, andwherein the means for analyzing the second image comprises means foridentifying a region of the second image having at least one of highintensity or high contrast relative to surrounding regions of the secondimage.

Example 33 includes the subject matter of any of Examples 30-32, andwherein the means for determining the physical location comprises meansfor determining the physical location of the light source based on thesecond image and a third image of the real-world environment thatincludes the light source.

Example 34 includes the subject matter of any of Examples 30-33, andwherein the means for determining the physical location comprises meansfor determining a distance of the light source from the mobile computedevice by performing triangulation based on the second image and thethird image.

Example 35 includes the subject matter of any of Examples 30-34, andwherein the means for determining the physical location comprises meansfor approximating a distance of the light source from the mobile computedevice to a predefined distance.

Example 36 includes the subject matter of any of Examples 30-35, andwherein the means for identifying the first corneal reflection and thesecond corneal reflection comprises means for analyzing the first imageto identify the user's eye.

Example 37 includes the subject matter of any of Examples 30-36, andwherein the means for identifying the first corneal reflection and thesecond corneal reflection comprises means for identifying a first regionand a second region of the first image having at least one of highintensity or high contrast relative to surrounding regions of the firstimage.

Example 38 includes the subject matter of any of Examples 30-37, andwherein the means for determining the first correspondence and thesecond correspondence comprises means for determining the firstcorrespondence and the second correspondence based on knowncharacteristics of at least one of the light source or the imagedisplayed on the display.

Example 39 includes the subject matter of any of Examples 30-38, and,further including means for determining the image displayed, at a pointin time at which the first image is captured, on the display of themobile compute device; wherein the means for determining the secondcorrespondence comprises means for determining the second correspondencebased on at least one characteristic of the image displayed.

Example 40 includes the subject matter of any of Examples 30-39, andwherein the means for determining the second correspondence comprisesmeans for determining the second correspondence based on at least one ofa size or color of the image displayed.

Example 41 includes the subject matter of any of Examples 30-40, andfurther including means for analyzing the first image to identify anedge of a pupil of the user; wherein the means for performing the gazetracking comprises means for performing the gaze tracking based furtheron the edge of the pupil of the user.

Example 42 includes the subject matter of any of Examples 30-41, andwherein the first camera has a field of view in a direction opposite afield of view of the second camera about the display.

The invention claimed is:
 1. A mobile compute device for gaze trackingby ambient light, the mobile compute device comprising: a display; acamera system comprising a first camera and a second camera, the camerasystem to capture (i) a first image of a user of the mobile computedevice with the first camera and (ii) a second image of a real-worldenvironment of the mobile compute device with the second camera; a lightanalysis module to determine a physical location of an ambient lightsource relative to the mobile compute device based on the second image;an eye analysis module to (i) identify a first corneal reflection and asecond corneal reflection in an eye of the user and (ii) determine,based on the physical location, a first correspondence between the firstcorneal reflection and the ambient light source and a secondcorrespondence between the second corneal reflection and an imagedisplayed on the display; a gaze tracking module to perform gazetracking based on the first correspondence and the secondcorrespondence.
 2. The mobile compute device of claim 1, wherein thelight analysis module is to (i) analyze the second image to determine animage location of the ambient light source captured in the second imageand (ii) determine a direction of the ambient light source relative tothe mobile compute device based on the image location; and wherein todetermine the physical location of the ambient light source comprises todetermine the physical location based on the determined direction. 3.The mobile compute device of claim 2, wherein to analyze the secondimage comprises to identify a region of the second image having at leastone of high intensity or high contrast relative to surrounding regionsof the second image.
 4. The mobile compute device of claim 2, wherein todetermine the physical location comprises to determine the physicallocation of the ambient light source based on the second image and athird image of the real-world environment that includes the ambientlight source.
 5. The mobile compute device of claim 4, whereindetermining the physical location comprises determining a distance ofthe ambient light source from the mobile compute device by performingtriangulation based on the second image and the third image.
 6. Themobile compute device of claim 2, wherein to determine the physicallocation comprises to approximate a distance of the ambient light sourcefrom the mobile compute device to a predefined distance.
 7. The mobilecompute device of claim 1, wherein to identify the first cornealreflection and the second corneal reflection comprises to identify afirst region and a second region of the first image having at least oneof high intensity or high contrast relative to surrounding regions ofthe first image.
 8. The mobile compute device of claim 1, wherein todetermine the first correspondence and the second correspondencecomprises to determine the first correspondence and the secondcorrespondence based on known characteristics of at least one of theambient light source or the image displayed on the display.
 9. Themobile compute device of claim 8, further comprising a display module todetermine the image displayed, at a point in time at which the firstimage is captured, on the display of the mobile compute device; whereinto determine the second correspondence comprises to determine the secondcorrespondence based on at least one characteristic of the imagedisplayed.
 10. The mobile compute device of claim 9, wherein todetermine the second correspondence comprises to determine the secondcorrespondence based on at least one of a size or color of the imagedisplayed.
 11. The mobile compute device of claim 1, wherein the eyeanalysis module is to analyze the first image to identify an edge of apupil of the user; and wherein to perform the gaze tracking comprises toperform the gaze tracking based further on the edge of the pupil of theuser.
 12. The mobile compute device of claim 1, wherein the first camerahas a field of view in a direction opposite a field of view of thesecond camera about the display.
 13. One or more non-transitory,machine-readable storage media comprising a plurality of instructionsstored thereon that, in response to execution by a compute device, causethe compute device to: capture, by a first camera of the compute device,a first image of a user of the compute device; capture, by a secondcamera of the compute device different from the first camera, a secondimage of a real-world environment of the compute device; determine aphysical location of an ambient light source relative to the computedevice based on the second image; identify a first corneal reflectionand a second corneal reflection in an eye of the user captured in thefirst image; determine, based on the physical location, a firstcorrespondence between the first corneal reflection and the ambientlight source and a second correspondence between the second cornealreflection and an image displayed on a display of the compute device;and perform gaze tracking based on the first correspondence and thesecond correspondence.
 14. The one or more non-transitory,machine-readable storage media of claim 13, wherein the plurality ofinstructions further cause the compute device to: analyze the secondimage to determine an image location of the ambient light sourcecaptured in the second image; and determine a direction of the ambientlight source relative to the compute device based on the image location;wherein to determine the physical location of the ambient light sourcecomprises to determine the physical location based on the determineddirection.
 15. The one or more non-transitory, machine-readable storagemedia of claim 14, wherein to analyze the second image comprises toidentify a region of the second image having at least one of highintensity or high contrast relative to surrounding regions of the secondimage.
 16. The one or more non-transitory, machine-readable storagemedia of claim 14, wherein to determine the physical location comprisesto determine the physical location of the ambient light source based onthe second image and a third image of the real-world environment thatincludes the ambient light source.
 17. The one or more non-transitory,machine-readable storage media of claim 16, wherein to determine thephysical location comprises to determine a distance of the ambient lightsource from the compute device by performing triangulation based on thesecond image and the third image.
 18. The one or more non-transitory,machine-readable storage media of claim 15, wherein to determine thephysical location comprises to approximate a distance of the ambientlight source from the compute device to a predefined distance.
 19. Theone or more non-transitory, machine-readable storage media of claim 13,wherein to determine the first correspondence and the secondcorrespondence comprises to determine the first correspondence and thesecond correspondence based on known characteristics of at least one ofthe ambient light source or the image displayed on the display.
 20. Theone or more non-transitory, machine-readable storage media of claim 13,wherein the first camera has a field of view in a direction opposite afield of view of the second camera about the display.
 21. A method forgaze tracking by ambient light, the method comprising: capturing, by afirst camera of a mobile compute device, a first image of a user of themobile compute device; capturing, by a second camera of the mobilecompute device different from the first camera, a second image of areal-world environment of the mobile compute device; determining, by themobile compute device, a physical location of an ambient light sourcerelative to the mobile compute device based on the second image;identifying, by the mobile compute device, a first corneal reflectionand a second corneal reflection in an eye of the user captured in thefirst image; determining, by the mobile compute device and based on thephysical location, a first correspondence between the first cornealreflection and the ambient light source and a second correspondencebetween the second corneal reflection and an image displayed on adisplay of the mobile compute device; and performing, by the mobilecompute device, gaze tracking based on the first correspondence and thesecond correspondence.
 22. The method of claim 21, further comprising:analyzing, by the mobile compute device, the second image to determinean image location of the ambient light source captured in the secondimage; and determining, by the mobile compute device, a direction of theambient light source relative to the mobile compute device based on theimage location; wherein determining the physical location of the ambientlight source comprises determining the physical location based on thedetermined direction.
 23. The method of claim 21, wherein determiningthe first correspondence and the second correspondence comprisesdetermining the first correspondence and the second correspondence basedon known characteristics of at least one of the ambient light source orthe image displayed on the display.
 24. The method of claim 23, furthercomprising determining the image displayed, at a point in time at whichthe first image is captured, on the display of the mobile computedevice; wherein determining the second correspondence comprisesdetermining the second correspondence based on at least onecharacteristic of the image displayed.
 25. The method of claim 21,further comprising analyzing, by the mobile compute device, the firstimage to identify an edge of a pupil of the user; wherein performing thegaze tracking comprises performing the gaze tracking based further onthe edge of the pupil of the user.